The present invention relates to an imaging apparatus that receives light from an imaging region by a light-receiving element array through an optical filter in which at least equal to or more than one type of a polarization filter part, or a color separation filter part is periodically arranged, and outputs an imaged image based on an image signal thus obtained, a vehicle system having the imaging apparatus, and an image-processing method.
This type of imaging apparatus is provided in a mobile object controller that performs mobile control of a mobile object such as a vehicle, a vessel, an air plane, or an industrial robot, an information supplier that supplies beneficial information to an operator (a driver) of the mobile object, and the like, and is widely used for an object identification operation.
In particular, for example, it is known that one is used for a vehicle travel support system such as ACC (Adaptive Cruise Control) for reduction of an operating load of a driver of a vehicle, and so on.
In such a vehicle travel support system, various functions are achieved such as an automatic brake function, or an alarm function that prevents a driver's vehicle from crashing into an obstacle or the like, and reduces an impact at the time of crashing, a vehicle speed-adjusting function that maintains a distance between a vehicle in front, a lane keep-assisting function that prevents a driver's vehicle from deviating from a traveling lane. Therefore, it is important to analyze an imaged image imaged around the driver's vehicle, and obtain various types of information showing a state around the driver's vehicle as precisely as possible. As the information showing the state around the driver's vehicle, for example, there are position information of various objects such as obstacles existing around the driver's vehicle, other vehicles, lane lines (white lines, Botts' dots, and the like), road-surface elements such as manhole covers and so on, road-side constructions such as guard rails and so on, road surface information showing whether a road surface is dry or wet, sunlight information showing whether it is a sunny side or a shady part, and the like.
A general imaging apparatus detects intensity of light (brightness information) from an imaging region, and obtains an imaged image based on the brightness information.
On the other hand, in order to detect (perform sensing) a shape, a material, a surface state of an object existing in the imaging region, an imaging apparatus that is capable of obtaining an imaged image (polarization information image) in which polarization information is reflected has attracted attention in recent years.
Such an imaging apparatus uses various types of partial polarization occurring by reflected light (specularly-reflected light, or diffusely-reflected light) from an object to which specifically-polarized light or non-polarized natural light is emitted, due to a geometric reason such as a direction of a surface of the object, an imaging position with respect to the object, or the like, a surface material of the object, and the like. With such an imaging apparatus, a two-dimensional distribution is obtained of a plurality of polarization components that are included in reflected light from an object in an imaging region, and their polarization directions are different to each other. And, by comparing a difference in magnitude among the polarization components included in light from the object in the imaging region, it is possible to obtain the position information of the various objects, the road surface information, the sunlight information, and the like that are difficult to obtain only from the brightness information with higher accuracy.
For example, Japanese Patent Application Publication number H11-175702 discloses an imaging apparatus in which an image sensor that images an image via a polarization filter that transmits only a vertical polarization component, and an image sensor that images an image via a horizontal polarization filter that transmits only a horizontal polarization component are arranged in parallel. In the imaging apparatus, from the former image sensor, an image signal of a vertical polarization image that expresses a two-dimensional distribution of the vertical polarization component included in the reflected light from the object in the imaging region is outputted, and from the latter image sensor, an image signal of a horizontal polarization image that expresses a two-dimensional distribution of the vertical polarization component included in the reflected light from the object in the imaging region is outputted.
In the imaging apparatus, as for the image signals of those of the vertical polarization image and the horizontal polarization image, after correcting displacement of a position due to parallax, a polarization ratio (index value) that indicates a ratio of intensity of vertical polarization to intensity of horizontal polarization per pixel is calculated, and a polarization ratio image (an index value image) in which the polarization ratio is taken as a pixel value is obtained.
By analyzing the polarization ratio image imaged by the imaging apparatus disclosed in Japanese Patent Application publication number H11-175702, it is possible to obtain the above-described various types of information such as the position information of the various objects, and the like. Additionally, from not only the polarization ratio image, but also an index value image having a pixel value based on an index value that indicates a difference of magnitude among polarization components included in light from each position in an imaging region, for example, an index value image in which a difference value among those polarization components, or a value that expresses a ratio of the difference value of those polarization components to a total value of those polarization components is taken as the index value, it is possible to obtain the above-described various information.
In order to analyze such an index value image and obtain various types of information in an imaging region with high accuracy, generally, contrast of the index value image (in other words, resolution of an index value) is important. However, in a conventional index value image, it is not possible to obtain sufficient contrast due to transmittance characteristics of a polarization filter. That is, there is a problem in that in order to accurately obtain various types of information in an imaging region, resolution of an image is insufficient. Hereinafter, regarding the problem, a case where a differential polarization degree that shows a ratio of a difference value of those polarization components to a total value of a vertical polarization component and a horizontal polarization component is taken as an index value will be explained specifically.
Generally, even if a transmittance characteristic of a vertical polarization filter (hereinafter, also referred to as “a P-polarization filter”) that transmits a vertical polarization component (hereinafter, also referred to as “a P-polarization component”) is high, 100% of the vertical polarization component of incident light is not transmitted, and it does not cut 100% of a horizontal polarization component (hereinafter, also referred to as “an S-polarization component”) of the incident light. A horizontal polarization filter (hereinafter, also referred to as “an S-polarization filter”) that transmits the horizontal polarization component does not transmit 100% of the horizontal polarization component of the incident light, and it does not cut 100% of the vertical polarization component. The following Table 1 shows a specific example of transmittance characteristics of actual S-polarization filter and P-polarization filter. The specific example shows a case where transmittance is measured when light of only the S-polarization component (100% S-polarization light), and light of only the P-polarization component (100% P-polarization light) are incident onto the S-polarization filter and the P-polarization filter, respectively. Note that each value in a bracket in the following Table 1 shows transmittance in a case where each filter has an ideal transmittance characteristic.
TABLE 1P-POLARIZATIONS-POLARIZATIONFILTERFILTER100% P-POLARIZATION78% (100%)32% (0%)LIGHT100% S-POLARIZATION17% (0%)64% (100%)LIGHT
As shown by using the brackets in the above Table 1, as for the P-polarization filter, it is ideal that transmittance of the P-polarization component be 100%, and transmittance of the S-polarization component be 0%. However, in fact, since the transmittance of the P-polarization component is 78%, the P-polarization component is not completely transmitted. Additionally, since the transmittance of the S-polarization component is 17%, the S-polarization component is not cut completely. This similarly applies to a case of the S-polarization filter.
In an imaging apparatus that obtains a differential polarization degree image (an index value image) having a pixel value based on a differential polarization degree by using a P-polarization filter and an S-polarization filter having the transmittance characteristics shown in the above Table 1, in the case where each of 100% P-polarization light and 100% S-polarization light having intensity of light equivalent to a maximum value of an effective amount of light received by an image sensor (light-receiving element) is incident, each signal value of an image signal outputted by the image sensor is shown in the following Table 2. Note that the signal value is 10-bit data, and a minimum value of the effective amount of the received-light is 0, and a maximum value of the effective amount of the light received is 1023. Additionally, each value in a bracket in the following Table 2 shows a signal value of an image signal that is outputted in a case where each filter has an ideal transmittance characteristic.
TABLE 2OUTPUT VALUE OFOUTPUT VALUE OFP-POLARIZATIONS-POLARIZATIONFILTERFILTER100%1023 * 78/100 = 7971023 * 32/100 = 327 (0)P-POLARIZATION(1023)LIGHT100%1023 * 17/100 = 173 (0)1023 * 64/100 = 654S-POLARIZATION(1023)LIGHT
As for a signal value outputted from an image sensor that receives the 100% P-polarization light via the P-polarization filter, an ideal signal value is 1023; however, transmittance of P-polarization light of the P-polarization filter is 78% as shown in the above Table 1, and therefore the signal value is 797. Likewise, as for a signal value outputted from an image sensor that receives the 100% S-polarization light via the S-polarization filter, an ideal signal value is 0; however, due to transmittance of S-polarization light of the S-polarization filter, the signal value is 327. Additionally, as for a signal value outputted from an image sensor that receives the 100% P-polarization light via the S-polarization filter, an ideal signal value is 0; however, due to transmittance of the P-polarization light of the S-polarization filter, the signal value is 173. Furthermore, as for a signal value outputted from an image sensor that receives the 100% S-polarization light via the S-polarization filter, an ideal signal value is 1023; however, due to transmittance of the S-polarization light of the S-polarization filter, the signal value is 654.
The following Table 3 shows results of each of a case where the 100% P-polarization light is incident and a case where the 100% S-polarization light is incident, and from signal values shown in the above Table 2, a total value (P+S) of a signal value P of the image signal via the P-polarization filter and a signal value S of the image signal via the S-polarization filter, and a difference value (P−S) in which the signal value S is subtracted from the signal value P are calculated. The total value (P+S) is never a value over 1023 in principle. This is because no matter how intense light is incident, a total of the P-polarization component and the S-polarization component cannot be over 100%. As shown in the following Table 3, the total value (P+S) in the case where the 100% P-polarization light is incident is a value over 1023, and therefore, here, it is rounded to 1023. Note that each value in brackets in the following Table 3 shows the total value (P+S) or the difference value (P−S) in the case where each filter has the ideal transmittance characteristic.
TABLE 3P + SP − S100%797 + 327 = 1023 (1023)797 − 327 = 470 (1023)P-POLARIZATIONLIGHT100%173 + 654 = 827 (1023)173 − 654 = −481S-POLARIZATION(−1023)LIGHT
The following Table 4 shows a differential polarization degree calculated from the total value (P+S) and the difference value (P−S) shown in the above Table 3. Note that each value in brackets in the following Table 4 shows a differential polarization degree in the case where each filter has the ideal transmittance characteristic.
TABLE 4DIFFERENTIAL POLARIZATIONDEGREE100% P-POLARIZATION LIGHT470/1023 ≈ 0.46 (1)100% S-POLARIZATION LIGHT−481/827 ≈ −0.58 (−1)
The differential polarization degree is an index value that takes values in a range between equal to or more than −1 and less than or equal to 1 in the case where each filter has the ideal transmittance characteristic. However, an actual filter does not have such an ideal transmittance characteristic, but has the transmittance characteristic shown in the above Table 1. Therefore, in a case of using the actual filter that has the transmittance characteristic shown in the above Table 1, as shown in the above Table 4, a range that can be taken by the differential polarization degree is a range between equal to or more than −0.58 and less than or equal to 0.46, and it is narrower than the ideal one.
In a case of creating a differential polarization degree image of 1024 tones constituted by a 10-bit pixel value, the differential polarization degree (range between equal to or more than −1 and less than or equal to 1) is scaled to a pixel value of the differential polarization degree image (range between equal to or more than 0 and less than or equal to 1023). At this time, in a case where the range that can be taken by the differential polarization degree is a range between equal to or more than −0.58 and less than or equal to +0.46 as described above, and a range that can be taken by the pixel value of the differential polarization degree image is a range between equal to or more than 215 and less than or equal to 747, only a differential polarization degree image of 533 tones can be created. Therefore, contrast of the differential polarization degree image is low.
With respect to a problem in that the contrast of the differential polarization degree image is thus low, and image analysis accuracy is insufficient, the following solution is considered.
That is, a method is considered that performs an operation in which with respect to an image of a differential polarization degree image for one frame obtained by imaging an imaging region, a range of a pixel value included in the differential polarization degree image is expanded to a maximum range (range between equal to or more than 0 and less than or equal to 1023) that can be taken by the pixel value of the differential polarization degree image. In this method, the range of the pixel value included in the differential polarization degree image for one frame is defined, and magnification for expanding the pixel value included in the differential polarization degree image for one frame to the maximum range (range of an ideal index value) that can be taken by the pixel value of the differential polarization degree image is calculated, and by use of the calculated magnification, the range of the pixel value is expanded. Thus, a range of a pixel value after correction by the above expansion operation is expanded more than a range of a pixel value before correction, within a limit of not over the range of the ideal index value. Therefore, an image of a differential polarization degree having the pixel value after correction has contrast higher than an image of a differential polarization degree having a pixel value before correction.
However, regardless of the range of the pixel value before correction, when magnification for expanding the range of the pixel value to the maximum range that can be taken by the pixel value of the differential polarization degree image is calculated, and the expansion operation is performed by the magnification, the following problem occurs. As described above, by the transmittance characteristic that each filter has, the range that can be taken by the pixel value before correction is limited to a predetermined range (range of an effective index value) calculated from the transmittance characteristic (limited to an effective range between equal to or more than 215 and less than or equal to 747, in an example of the above Table 4). Therefore, if the expansion operation by using effective magnification in order to enlarge at least this effective range (215-747) to an ideal range (0-1023) is performed, an appropriate pixel value in the effective range (pixel value before correction) is not a pixel value that is out of the ideal range (pixel value after correction) after performing the expansion operation, and an excess expansion operation is not performed. Accordingly, in principle, at least, by using effective magnification corresponding to the effective range (215-747) calculated from the transmittance characteristic of the filter, the excess expansion operation is not performed, and it is possible to increase the contrast of the differential polarization degree image.
However, in fact, where an image signal includes noise, and so on, there is a case where the pixel value (pixel value before correction) that is out of the effective range (215-747) is calculated.
In this case, magnification calculated based on the range of the pixel value before correction becomes lower than the effective magnification. As a result, in principle, even though it is possible to perform an expansion operation at equal to or more than the effective magnification at minimum, the magnification is suppressed by the existence of an inappropriate pixel value that is out of the effective range. Therefore, the expansion operation of an appropriate pixel value (pixel value before correction) included in the effective range is insufficient, and a problem occurs in that an index value image of high contrast is not able to be obtained.